Control system, control device, image processing device, and control method

ABSTRACT

A control system configured to use a database and a data storage area includes a control device for controlling a process executed on an object; and an image processing device arranged in association with the control device for taking an image of the object and processing image data acquired from taking the image of the object. The control device and the image processing device may work independently or collaboratively to send the database at least one of an attribute value or results information, and designation information in association with each other for the same object, the attribute value managed by the control device and corresponding to any attribute defined in the database, the results information representing a process result from the image processing device, and the designation information specifying a storage destination in the data storage area for the image data acquired from taking an image of the object.

TECHNICAL FIELD

The present invention relates to a control system configured to use adatabase and a data storage area, a control device and image processingdevice suited thereto, and a control method used in the control system.

BACKGROUND ART

Typically, controlled objects such as the machines and equipment used atmany production sites are controlled by control devices such asprogrammable controllers (also referred to below as a programmable logiccontroller, PLC). The information used to manage a control systemcontaining these kinds of control devices are largely collected andanalyzed on higher-level computers.

More specifically, there is a need to be able to perform fault analysisand state analysis during the manufacturing process. To address thisneed, Japanese Unexamined Patent Application Publication No. 2011-253469(Patent Document 1) discloses a product inspection information recordingsystem that allows swift and easy ways to understand issues with theproducts even with a plurality of inspection parameters related to theproduct. The product inspection information recording system isconfigured such that the inspection results data includes weight data aswell as foreign-particle presence data for the product.

TECHNICAL PROBLEM

The information used to manage a control system may include, forinstance identification information and various inspection resultsrelated to the controlled object. The inspection results may include theimage data acquired by taking an image of an object (e.g., a work piece)being manipulated by the control object. From a production managementperspective, which aims to reduce the percentage defects and the like,this image data acquired by taking an image of the object is alsopreferably included as a management parameter.

A database is generally configured for the retention and management oftext based information, however it is not as simple for a database tohandle image data. Further, a control system that manages the productionand inspection of multiple work pieces needs to collect and analyze ahuge amount of information. A generic production site usually manages aplurality of stages such as a fabrication stage and an inspection stage.When managing these production stages in their entirety, it is alsonecessary to mutually map the information acquired during these stageswith the respective stages.

Therefore, the desire is for a configuration that allows more effectiveanalyses of the various pieces of information managed by the controlsystem and the image data from a corresponding object.

SUMMARY

One aspect of the present invention provides a control system configuredto use a database and a data storage area. The control system includes acontrol device for controlling a process executed on an object; and animage processing device arranged in association with the control devicefor taking an image of the object and processing image data acquiredfrom taking the image of the object. The image data acquired by theimage processing device may be configured for storage in a data storagearea. The control device and the image processing device may workindependently or collaboratively to send the database at least one of anattribute value or results information, and designation information inassociation with each other for the same object, the attribute valuemanaged by the control device and corresponding to any attribute definedin the database, the results information representing a process resultfrom the image processing device, and the designation informationspecifying a storage destination in the data storage area for the imagedata acquired from taking an image of the object.

The control device may provide the image processing device with at leastone of identification information and location information for eachpiece of image data acquired by the image processing device asdesignation information, where the identification information is createdwhen storing the image data in the data storage area, and the locationinformation represents the location where the image data is stored.

Alternatively, the image processing device in the control system may beconfigured to store each piece of image data acquired in the datastorage area in accordance with a preliminarily determined rule, andprovide the control device the designation information corresponding toeach piece of image data stored.

The control device in the control system may be further configured tosend the database at least one of information managed by the controldevice and information managed by the image processing device togetherwith designation information for specifying the image data stored in thedata storage area.

Alternatively the image processing device in the control system may befurther configured to store each piece of image data acquired in thedata storage area, and provide the control device with time informationrepresenting the timing at which the image data was stored.

The control device in control system may be further configured to sendthe database at least one of information managed by the control deviceand information managed by the image processing device together with thetime information; and a process is executed in the database to create amapping for the corresponding image data on the basis of the timeinformation.

Another aspect of the present invention proposes a control deviceconfigured to communicate with an external storage device and an imageprocessing device. The control device includes a controller, and astorage unit used to store a user program. The controller executes theuser program to control processes executed on an object. The controllerconfigured to execute a process to store machining data related to thecontrol associated with the object in the storage unit; a process tosend the image processing device designation information specifying astorage destination in the external storage device where the imageprocessing device is to store the image data; and a process to send theexternal storage device the machining data and the designationinformation for the object corresponding to the machining data.

Another aspect of the present invention proposes an image processingdevice configured to communicate with an external storage device and acontrol device. The image processing device includes a controller, aninterface for acquiring image data by taking an image of an object, anda storage unit used for storing the image data. The controllerconfigured to execute a process initiating image processing of the imagedata; a process sending the control device results generated from theimage processing; and a process sending the external storage device theacquired image data in accordance with designation information from thecontrol device specifying the storage destination of the image data onthe external storage device.

The image processing device may be further configured to store eachpiece of image data acquired in the external storage device, and providethe control device with time information representing the timing atwhich the image data was stored.

Another aspect of the present invention provides a control method in acontrol system configured to use a database and a data storage area. Thecontrol method including steps of: controlling a process a controldevice executes on an object; executing a process on image data acquiredfrom an image processing device arranged in association with the controldevice taking an image of the object; storing the image data acquired bythe image processing device in the data storage area; and causing thecontrol device and the image processing device to work independently orcollaboratively to send the database at least one of an attribute valueor results information, and designation information in association witheach other for the same object, the attribute value managed by thecontrol device and corresponding to any attribute defined in thedatabase, the results information representing a process result from theimage processing device, and the designation information specifying astorage destination in the data storage area for image informationacquired from taking an image of the object.

Effects

Therefore, the present invention provides for more effective analyses ofthe various pieces of information managed by the control system and theimage data from a corresponding object.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a configuration of aproduction system including a control system according to theembodiments;

FIG. 2 illustrates an example of analysis results output from ananalysis device in a production system according to the embodiments;

FIG. 3 illustrates an example of a data structure in a database in aproduction system according to the embodiments;

FIG. 4 is a schematic diagram illustrating a configuration of a PLCaccording to the embodiments;

FIG. 5 is a schematic diagram illustrating a configuration of a visualsensor according to the embodiments;

FIG. 6 is a schematic diagram illustrating a configuration of aninformation gathering device according to the embodiments;

FIG. 7 is a sequence diagram illustrating a processing procedure in theproduction system according to a first embodiment;

FIG. 8 is a sequence diagram illustrating a processing procedure in theproduction system according to a second embodiment; and

FIG. 9 is a sequence diagram illustrating a processing procedure in theproduction system according to a third embodiment.

DETAILED DESCRIPTION

Embodiments of the present invention will be described in detail withreference to the drawings. The same or corresponding elements within thedrawings will be given the same reference numerals and the explanationstherefor will not be repeated.

A. Overall Configuration

First, an overall configuration of a control system according to theembodiments is described. FIG. 1 is a schematic diagram illustrating aconfiguration of a production system SYS including a control system 1according to the embodiments.

Referring FIG. 1, production system SYS includes a control system 1, aninformation gathering device 300, and an analysis device 400. Thecontrol system 1 includes a programmable controller (referred to belowas a PLC, 100), and a visual sensor 200. Assume that these devices areconnected via a network 30.

The PLC 100 is a specific example of a typical implementation of acontrol device that controls the processing executed on a workpiece(hereafter, “object”) handled by the controlled object. This manner ofcontrol device is not limited to a PLC, and may be implemented in ageneral purpose computer, or in dedicated hardware.

In the production system SYS illustrated in FIG. 1, a robot 10 carriesout a fabrication process (hereafter, “Fab Process 1”) on workpieces 4continuously transported thereto via a conveyor 2. Thereafter, a robot20 carries out another fabrication process (hereafter, “Fab Process 2”)on the workpieces 4. The PLC 100 controls the respective fabricationprocesses carried out by the robots 10 and 20, as well as transportationby the conveyor 2.

The visual sensor 200 determines whether or not the fabrication carriedout by the robots 10 and 20 on a workpiece was suitable. Thedetermination process carried out by the visual sensor 200 is referredto below as the “Inspection Process”. Namely, the visual sensor 200 is aspecific example of a typical image processing device setup inassociation with the PLC 100 (control device) to process the image data50 acquired by taking an image of an object being handled, i.e., aworkpiece 4. The image processing carried out in the visual sensor 200includes, for instance, detecting the position of the workpiece 4itself, determining whether the workpiece 4 is good or defective, andinspecting or tracking a specific section extracted from an image of theworkpiece 4.

The information gathering device 300 and the analysis device 400provided in the control system 1 are used in production management.These devices collect the information the control system 1 manages,analyze the information collected, and output the results to a user, asnecessary.

The information gathering device 300 contains a database 350 and a datastorage area 360. The database retains and manages the variousinformation transmitted thereto from the control system 1. Data in thedatabase 350 is updated via a database engine (DB engine, 352).Information from the control system 1 is typically transmitted as aquery (Query, 40) which is composed of a character string representing aprocessing request.

The data storage area 360 may be provided as a file server, or the like,to receive and store the image data 50 transmitted thereto from thecontrol system 1. The image data 50 acquired by the visual sensor 200taking an image of a workpiece 4 may be transmitted directly orindirectly therefrom to the information gathering device 300 in thecontrol system 1. That is, the data storage area 360 may be configuredto store the image data 50 acquired by the visual sensor 200 (imageprocessing device). In other words, the information gathering device 300acts as an external storage device.

The control system 1 is thusly configured to use a database and a datastorage area.

FIG. 1 illustrates a typical configuration example where the informationgathering device 300 contains the database 350 and the data storage area360; however, the information gathering device 300 is not limited tothis configuration. Namely, the database 350 and the data storage area360 may be packaged as different devices, or may be packaged in a singledevice. Moreover, the database 350 may be provided as the databasefunctions stored in each of a plurality of devices that are integratedvirtually. The data storage area 360 may be similarly implemented on aplurality of devices. Alternatively, the database 350 and the datastorage area 360 may be packaged with at least one of the PLC 100 andthe visual sensor 200.

Finally, the analysis device 400 may analyze the information stored inthe information gathering device 300, and the image data 50 stored inthe data storage area 360 in association with each other and output theresults of the analysis to thereby assist with fault analysis or stateanalysis during production.

B. Analysis

Next is an overview of the analysis provided by the production systemSYS according to the embodiments. FIG. 2 illustrates an example of theanalysis results output from the analysis device 400 in the productionsystem SYS according to the embodiments.

Referring to FIG. 2, the analysis device 400 classifies the image data50 stored in the data storage area 360 on the basis of the informationassociated therewith (such as, the attribute values defined for aplurality of different attributes), and displays the classified imagedata 50 in association with the attribute value used in theclassification thereof. For instance, a plurality of attributes may beassociated with each piece of image data 50, and assigned attributenames such as “Serial No.,” “Device No.,” “Part No.,” “Process Date”,“Inspection Date”, “Inspection Results from Inspection Process(Pass/Fail)”, and the like. In the analysis results illustrated in FIG.2, the image data 50 having an inspection result from the inspectionstage (attribute value) of “Fail” are grouped by “Part No.”, with theimage data 50 belonging to each of the groups also displayed. Amagnified view may be provided for a selected image data 50 once theimage data 50 is selected.

The analysis device may be further configured such that selecting adesired attribute (attribute name) associated with the image data 50groups and displays the image data 50 according to the selection. FIG. 2illustrates an example where the image data is grouped according to“Inspection Results from Inspection Process (Pass/Fail)”, and “PartNo.”; however, an additional attribute may be used for grouping. Forinstance, the analysis results illustrated in FIG. 2 may be furthergrouped by “Process Date” to allow analysis of the chronologicalvariations in the state of the production process. In this case, theimage data 50 may be displayed in relation to two-dimensionalcoordinates defined by the “Part No.” and the “Process Date”.

The production system SYS according to the embodiments is capable ofproviding analysis from the perspective of a desired attribute inrelation to the multitude of image data 50 taken from the productionsite in this manner, and thus facilitate understanding the status andtrends at the production site. Additionally, as illustrated in FIG. 2the image data 50 is displayed as the analysis result, and thereforeallows for diverse analysis. Moreover, data mining techniques may beadopted to facilitate fault analysis during production using theplurality of image data 50.

C. Database

Next is described a data structure in the database used for implementingthe above described kind of analysis.

The control system 1 according to the embodiments provides a foundationfor analysis using the above-described plurality of image data 50. Thatis, mappings must be automatically generated between the information inthe information gathering device 300 to carry out analysis using theinformation from the control system 1 (query 40) and the image data 50.Therefore, in the control system 1, the PLC 100 (control device) and thevisual sensor 200 (image processing device) work independently orcollaboratively to transmit at least one of information related to anattribute or results information, and designation information to thedatabase 350 in association with each other for the same workpiece 4(object). The information related to the attribute is managed by the PLC100 and corresponds to any attribute defined in the database 350; theresults information represents a process result from the visual sensor200, and the designation information specifies a storage destination inthe data storage area 360 for the image data 50 acquired from taking animage of the work piece 4. The visual sensor 200 may transmit the imagedata 50 to the information gathering device 300 directly or indirectlyto store the image data 50 thereon. Whereas, for each workpiece 4, thedatabase 350 stores the information representing where the image data 50transmitted from the visual sensor 200 is located. Hereby, referencingthe database 350, the system may uniquely specify which of any of thelocations in the data storage area 360 holds the image data 50corresponding to each of the workpieces 4.

FIG. 3 illustrates an example of a data structure in a database 350 in aproduction system SYS according to the embodiments. A single record isassociated with a single workpiece 4 in the data structure illustratedin FIG. 3. Note that FIG. 3 illustrates an individual table structurewherein a plurality of columns is provided per single record; however ifa relational database is adopted, a plurality of tables may be mapped toeach other.

Referring to FIG. 3, the table in the database 350 defines columns 3501,3502, 3503, 3504, 3505, 3506 corresponding to “Serial No.,” “DeviceNo.,” “Part No.,” “Process Date,” “Inspection Date,” “Inspection Resultsfrom Inspection Process (Pass/Fail)” respectively. These columnscorrespond to information stored in at least one of the PLC 100 and thevisual sensor 200. Another column 3507 stores a character stringrepresenting the location of the image data 50 obtained by the visualsensor 200 taking an image of each of the workpieces 4. Referencing thiskind of table, the information in each record (i.e., each workpiece 4)and the corresponding image data 50 may be uniquely mapped.

After describing the configuration of a device included in theproduction system SYS below, various implementations are described whichare used to map the workpieces and the image data 50 as illustrated inFIG. 3.

D. Configuration of PLC 100

Next, the configuration of a PLC 100 according to the embodiments isdescribed. FIG. 4 is a schematic diagram illustrating a configuration ofa PLC 100 according to the embodiments.

Referring to FIG. 4, the PLC 100 includes a CPU unit 110 that carriesout control processing, and one or more input output (10) units 130.These units are configured to mutually exchange data via an internal bus102. Additionally, a power unit (not shown) supplies power of theappropriate voltage to the units.

The CPU unit 110 contains a chipset 112, a processor 114, a main memory116 and non-volatile memory 118 which serve as a storage unit, a networkinterface 120, an internal bus controller 122, and a memory cardinterface 124. The chipset 112 and the other components are connected toeach other via various buses.

Typically, the chipset 112 and the processor 114 are configured based ona general purpose computer architecture. In more concrete terms, thechipset 112 exchanges internal data with the components connectedthereto, and generates the necessary command codes for the processor114. The chipset 112 includes a function for caching data and the likeobtained through the computations executed in the processor 114. Thechipset 112 sends the processor 114 the command codes sequentially inaccordance with an internal clock. The processor 114 interprets andexecutes these command codes in the order received. Any desiredconfiguration may be adopted for the processor 114 in place of a singleprocessor including a single core. For example, configurations includinga plurality of single-core processors, a plurality of single processors,or a plurality of multi-core processors, and the like may be adopted.

The CPU unit 110 includes a volatile main memory 116 and a non-volatilememory 118 as the storage means. The main memory 116 stores the variousprograms that should be run on the processor 114, and may also be usedas a working memory when the various programs are run. The main memory116 may also be used storage area that stores machining parameter data,or parameters in relation to the processing carried out on the object.More specifically, the main memory 116 stores machining data 116 a. Thedetails regarding the machining data 116 a are described later.

The non-volatile memory 118 permanently stores the operating system(OS), system program, user program, data definition information, loginformation, and the like. More specifically, the non-volatile memory118 stores a user program 118 a, a sequence command library 118 b, a DBaccess process library 118 c, and setup information 118 d. The userprogram 118 a includes the commands necessary for controlling thecontrolled object, and may be executed by the processor 114 periodicallyor due to an event. The sequence command library 118 b is called due tothe user program 118 a being run, and implements processing inaccordance with defined sequence commands. The user program 118 a alsoincludes commands for accessing the database 350 in the informationgathering device 300. When the processor 114 executes a command foraccessing the database, the DB access process library 118 c may becalled to transmit the necessary data to or to receive the necessaryfrom the database 350 via the network interface 120, and the like.

The setup information 118 d stored in the non-volatile memory 118contains information (network setting values, and the like) needed foraccessing the information gathering device 300 (the database 350 and thedata storage area 360).

The CPU unit 110 includes the network interface 120 and the internal buscontroller 122, which serve as communication means therefor. The networkinterface 120 provides for data exchange between the visual sensor 200and/or the information gathering device 300. The internal bus controller122 provides for data exchange with the IO unit 130 via the internal bus102.

The memory card interface 124 writes data to and reads data from amemory card 126 that is attachable to and removable from the CPU unit110.

The IO unit 130 executes input and output processing. More specifically,the IO unit 130 includes an IO module 134 and an internal bus controller132. The IO module 134 collects signals from a field and outputs datarepresenting the signal values to the CPU unit 110, and/or outputssignal values to the field in accordance with data from the CPU unit110. The various types of IO modules 134 may include a module for inputand outputting analog signals, a module for inputting and outputtingdigital signals, a module providing temperature adjustment functions,and a module providing position alignment functions.

The internal bus controller 132 communicates with the internal buscontroller 122 via the internal bus 102, to implement data exchangebetween the CPU unit 110 and the IO module 134.

E. Configuration of the Visual Sensor 200

Next, the configuration of a visual sensor 200 according to theembodiments is described. FIG. 5 is a schematic diagram illustrating aconfiguration of a visual sensor 200 according to the embodiments.

Referring to FIG. 5, the visual sensor 200 is connected to a camera 202and executes predetermined image processing on image data 50 generatedby the camera 202 taking an image of the object. More specifically, thevisual sensor 200 includes an IO controller 210, a system controller212, a processor 214, a main memory 216, a display unit 220, a harddrive 218, a camera interface 222, a memory card interface 224, an inputunit 228, a PLC interface 230, and a network interface 232. Thesecomponents are configured to mutually communicate data through thesystem controller 212.

The system controller 212 is respectively connected to the processor214, the main memory 216, the display unit 220, and the IO controller210 via buses. The system controller exchanges data with each of thesecomponents and controls the overall processing in the visual sensor 200.

The processor 214 exchanges programs and the like stored on the harddrive 218 with the system controller 212, and implements imageprocessing by executing the programs in a prescribed order.

The main memory 216 is a volatile storage device. In addition toprograms read from the hard drive 218, the main memory 216 stores imagedata acquired by the camera 202, data representing inspection resultsgenerated from imaging processing, work data, and the like.

The hard drive 218 is a non-volatile magnetic storage device. Inaddition to programs executed on the processor 214, the hard drive 218stores various setup values and the like. The programs installable onthe hard drive 218 may be run while stored on the memory card 226, andthe like as later described. The hard drive 218 may also store imagedata.

The display unit 220 displays various images in accordance with internalcommands from the system controller 212.

The IO controller 210 controls the exchange of data between the visualsensor 200 and the various devices connected thereto. More specifically,the IO controller 210 is connected the hard drive 218, the camerainterface 222, the input unit 228, the PLC interface 230, the networkinterface 232, and the memory card interface 224.

The camera interface 222 acts as an intermediary transmitting databetween the camera 202 and the processor 214. More specifically, thecamera interface 222 may be connected to one or more cameras 202, andcontains an image buffer 222 a for temporarily storing the image datafrom a camera 202.

The input unit 228 is an input device that is typically a keyboard, amouse, a touchscreen panel, or a dedicated console. The PLC interface230 acts as an intermediary transmitting data between the processor 214and the PLC 100. The network interface 232 acts as an intermediarytransmitting data between the processor 214 and other personal computersor server devices and the like (not shown). The network interface 232 istypically made up of an Ethernet (Registered Trademark), a UniversalSerial Bus (USB), or the like.

The memory card interface 224 writes data to and reads data from amemory card 226 that is attachable to and removable from the visualsensor 200.

F. Configuration of the Information Gathering Device 300

Next, the configuration of an information gathering device 300 accordingto the embodiments is described. FIG. 6 is a schematic diagramillustrating a configuration of an information gathering device 300according to the embodiments.

Referring to FIG. 6, the information gathering device 300 is a computerbased on a general purpose computer architecture. More specifically, theinformation gathering device 300 includes a CPU 302 for executing thevarious programs including an OS; a read-only memory (ROM) 304 forstoring a Basic Input Output System (BIOS) and various data; a RAMmemory 306 which provides a working area for storing data necessary forexecuting the programs in the CPU 302; and a hard drive (HDD) 308 forpermanently saving the programs and the like executed on the CPU 302.The hard drive 308 provides the database 350 and the data storage area360 illustrated in FIG. 1.

The information gathering device 300 further includes a keyboard 310 anda mouse 312 for accepting operational input from a user, and a monitor314 for presenting various information to the user. The informationgathering device 300 additionally includes a network interface 318 forcommunicating with the PLC 100 and the visual sensor 200.

Various processing programs that may be executed on the informationgathering device 300 are stored and run from an optical storage medium332. The various processing programs stored on the optical storagemedium are read by an optical disc reader 316, and transferred to andstored on the hard drive 308. Alternatively, the information gatheringdevice may be configured to download a program from an upper level hostcomputer, and the like through the network.

G. Configuration of the Analysis Device 400

The configuration of the analysis device 400 according to embodiments ofthe present invention is identical to the configuration of theinformation gathering device 300 illustrated in FIG. 6, and thus andexplanation thereof will not be repeated here.

H. First Embodiment

Described is a first embodiment where the PLC 100 actively instructs thevisual sensor 200 of the storage location of image data 50 obtained bytaking an image of an object, i.e., a workpiece 4. In the firstembodiment, the PLC 100 manages information gathered during Fab Process1 and Fab Process 2; this information is collectively referred to belowas “machining data”. On the other hand, the visual sensor 200 managesinformation related to the results from the inspection process; thisinformation is referred to below as “inspection data”. As abovedescribed, the visual sensor 200 takes an image of the workpiece 4 andgenerates image data 50.

The machining data 116 a (refer to FIG. 4) includes information such as“Serial No.,” “Device No.,” “Part No.,” “Process Date,” “Process DeviceSettings,” and the like. The inspection data includes information suchas “Serial No.,” “Device No.,” “Inspection Date,” “Inspection Resultsfrom Inspection Process (Pass/Fail),” “Measurement Value,” and the like.The “Measurement Value” may contain, for instance, a correlation valuerepresenting a degree of match with a model image.

In the first embodiment the PLC 100 generates a query containingmachining data held in the PLC 100 and inspection data output by thevisual sensor 200, and transmits the query to the database 350. Theimage data 50 generated by the visual sensor 200 is also transmitted forstorage to the data storage area 360. To facilitate mapping of theabove-described image data 50 illustrated in FIG. 3, the PLC 100instructs the visual sensor 200 of a storage destination for the imagedata 50 in advance, and includes the preliminarily instructed storagedestination information in the query.

That is, the PLC 100 (control device) provides the visual sensor 200(image processing device) with at least one of identificationinformation and location information for each piece of image data 50acquired by the visual sensor 200 (image processing device) asdesignation information, the identification information created whenstoring the image data 50 in the data storage area 360, and the locationinformation representing the location where the image data 50 is stored.The visual sensor 200 (image processing device) stores each piece ofimage data 50 acquired in the data storage area 360 in accordance withthe corresponding designation information sent from the PLC 100 (controldevice).

To provide the visual sensor 200 with the designation information(storage destination), the PLC 100 stores at least the setup information(setup information 118 d illustrated in FIG. 4) the visual sensor 200uses to access the database 350 and the data storage area 360. Thissetup information may contain the Internet Protocol (IP) address,network domain, folder name in the storage destination, and the like ofthe information gathering device 300. This setup information may also bepreliminarily created on the PLC 100 on the basis of information from auser controlling the information gathering device 300 and the like.Alternatively, the PLC 100 may send an inquiry message over a network30, and acquire the necessary setup information on the basis of aresponse from the information gathering device 300.

On the other hand, the PLC 100 (control device) transmits to thedatabase 350 at least one of the information managed by the PLC 100(typically, machining data) and the information managed by the visualsensor 200 (typically, inspection data), together with designationinformation for specifying the image data 50 stored in the data storagearea 360.

With this processing the image data 50 and the machining data, and theinspection data can be uniquely mapped to each other. A more concreteexample of the procedure is described with reference to a sequencediagram.

FIG. 7 is a sequence diagram illustrating a processing procedure in theproduction system SYS according to the first embodiment. FIG. 7illustrates a specific data exchange procedure between the PLC 100, thevisual sensor 200, and the information gathering device 300 (thedatabase 350 and the data storage area 360).

Referring to FIG. 7, first, tracking is established between the PLC 100and the visual sensor 200 (sequence SQ100). Tracking is a process ofidentifying the positional relationship (relative offset, and the like)between the PLC 100 and the visual sensor 200. For instance, a referenceworkpiece may be placed on a conveyor 2, and an amount of offsetcalculated between the timing at which the PLC 100 detects the arrivalof the reference workpiece, and the timing at which the visual sensor200 detects the arrival of the reference workpiece (alternatively anoffset time or the displacement of the conveyor 2 may be calculated);the result is then set as a tracking offset value. The data for the sameworkpiece 4 can be identified in the machining data maintained by thePLC 100, and the inspection data maintained by the visual sensor 200using this offset value. Actual operations begin once tracking isestablished.

The PLC 100 first detects the arrival of the workpiece 4 (sequenceSQ102), and initiates the execution of Fab Process 1 on the workpiece 4(sequence SQ104). The related machining data is generated and updatedwith the execution of Fab Process 1. Next, the PLC 100 initiates theexecution of Fab Process 2 on the workpiece 4 processed in Fab Process 1(sequence SQ106). The related machining data is generated and updatedwith the execution of Fab Process 2.

The PLC 100 then transmits designation information (identificationinformation and/or location information) to the visual sensor 200 forthe processed workpiece 4 (sequence SQ108). The designation informationis used for specifying the image data 50 stored in the data storage area360, while the identification information typically includes a filename. The location information contains the directory name or the foldername of where the image data 50 is stored. For example, when the imagedata 50 having different filenames for each individual workpiece 4 arestored in the same folder (e.g., as depicted by the file names in column3507 illustrated in FIG. 3 where the numbers in the column areincremented), transmitting the identification information containingonly the file name, for instance, is acceptable as the locationinformation containing the folder name is unnecessary. Whereas, when thefilename (e.g., image.jpg) is identical but the image data is stored indifferent folders for each of the workpieces 4, transmitting only thelocation information is acceptable as the identification information isunnecessary. Moreover, the image data stored in the data storage area360 may be specified using a Uniform Resource Locator (URL) and thelike; in this case, both the location information and the identificationinformation are transmitted. The visual sensor 200 sequentially storesthe designation information transmitted from the PLC 100.

The visual sensor 200 detects the arrival of the workpiece 4 (sequenceSQ110); the visual sensor 200 initiates image processing on theworkpiece 4 on detecting the arrival thereof (sequence SQ112). Thisimage processing includes an inspection process, such as whether theworkpiece 4 is good or defective. The inspection process is where thevisual sensor 200 acquires the image data 50 by taking an image of theworkpiece 4, and the inspection data.

The visual sensor 200 transmits the acquired image data 50 to the datastorage 360 in accordance with the designation information instructed inadvance from the PLC 100 (sequence SQ114). The data storage area 360stores the image data 50 transmitted from the visual sensor 200(sequence SQ116). At the same time, the visual sensor 200 sends the PLC100 the inspection data acquired from the inspection process (sequenceSQ118).

On receiving the inspection data from the visual sensor 200, the PLC 100uses the machining data acquired during Fab Process 1 and Fab Process 2,the inspection data from the visual sensor 200, and the designationinformation sent in advance to the visual sensor 200 to generate a query(sequence SQ120), and sends the query to the database 350 (sequenceSQ122). Note that all the information is not necessarily transmittedusing an individual query; the information may be divided among andtransmitted via a plurality of queries.

The database 350 (DB engine 352) updates the database on the basis of aquery transmitted from the PLC 100 (sequence SQ124).

Hereafter, the processing in the sequences SQ102 to SQ124 is repeatedfor each of the work pieces 4. Executing these processes builds the sortof database illustrated in FIG. 3, and uniquely maps the correspondingimage data 50 for each of the workpieces 4 to the machining data and theinspection data and vice versa.

I. Second Embodiment

Next is described the second embodiment wherein the visual sensor 200notifies the PLC 100 of the information specifying the stored image data50, and the PLC 100 creates the mappings for the information to betransmitted. In the second embodiment the PLC 100 still manages themachining data, and the visual sensor 200 still manages the inspectiondata.

The image data 50 generated by the visual sensor 200 is transmitted tothe data storage area 360 for storage. The PLC 100 also generates aquery containing machining data held in the PLC 100 and the inspectiondata output by the visual sensor 200, and transmits the query to thedatabase 350. The query the PLC 100 transmits to the database 350contains designation information specifying the image data 50 the visualsensor 200 requested to have stored.

That is, the visual sensor 200 (image processing device) stores eachpiece of image data 50 acquired in the data storage area 360 inaccordance with a preliminarily determined rule, and provides the PLC100 (control device) with the designation information corresponding toeach piece of image data 50 stored.

On the other hand, the PLC 100 (control device) sends the database 350at least one of the information managed by the PLC 100 (typically,machining data) and the information managed by the visual sensor 200(typically, inspection data), together with the designation informationspecifying the image data 50 stored in the data storage area 360.

With this processing the image data 50 can be uniquely mapped to themachining data and the inspection data and vice versa. A more concreteexample of the procedure is described with reference to a sequencediagram.

FIG. 8 is a sequence diagram illustrating a processing procedure in theproduction system SYS according to a second embodiment. FIG. 8illustrates a specific data exchange procedure between the PLC 100, thevisual sensor 200, and the information gathering device 300 (thedatabase 350 and the data storage area 360).

Referring to FIG. 8, tracking is first established between the PLC 100and the visual sensor 200 (sequence SQ200). Here the tracking isestablished in the same manner as in the above-described firstembodiment (sequence SQ100 in FIG. 7). Actual operations begin oncetracking is established.

The PLC 100 first detects the arrival of the workpiece 4 (sequenceSQ202), and initiates the execution of Fab Process 1 on the workpiece 4(sequence SQ204). The related machining data is generated and updatedwith the execution of Fab Process 1. Next, the PLC 100 initiates theexecution of Fab Process 2 on the workpiece 4 processed in Fab Process 1(sequence SQ206). The related machining data is generated and updatedwith the execution of Fab Process 2.

The visual sensor 200 detects the arrival of the workpiece 4 (sequenceSQ208); the visual sensor 200 initiates image processing on theworkpiece 4 on detecting the arrival thereof (sequence SQ210). Thisimage processing includes an inspection process, such whether theworkpiece 4 is good or defective. The inspection process is where thevisual sensor 200 acquires the image data 50 by taking an image of theworkpiece 4, and acquires the inspection data.

The visual sensor 200 transmits the acquired image data 50 to the datastorage 360 in accordance with predetermined rule (sequence SQ212). Thedata storage area 360 stores the image data 50 transmitted from thevisual sensor 200 (sequence SQ214). The predetermined rule may be, forinstance, incrementing the number included in a file name each time aworkpiece 4 arrives. Alternatively, designation information could beestablished on the basis of the kind of rules described in the abovefirst embodiment.

At the same time, the visual sensor 200 sends the PLC 100 thedesignation information (identification information and/or locationinformation) specifying the image data 50 stored in the data storagearea 360 (sequence SQ216), and also sends the PLC 100 the inspectiondata acquired from the inspection process (sequence SQ218).

On receiving the inspection data from the visual sensor 200, the PLC 100uses the machining data acquired during Fab Process 1 and Fab Process 2,the inspection data from the visual sensor 200, and the designationinformation sent in advance from the visual sensor 200 to generate aquery (sequence SQ220), and sends the query to the database 350(sequence SQ222). Note that all the information is not necessarilytransmitted using an individual query; the information may be dividedamong and transmitted via a plurality of queries.

The database 350 (DB engine 352) updates the database on the basis ofthe query transmitted from the PLC 100 (sequence SQ224).

Hereafter, the processing in the sequences SQ202 to SQ224 is repeatedfor each of the workpieces 4. Executing these processes builds the sortof database illustrated in FIG. 3, and uniquely maps the correspondingimage data 50 for each of the workpieces 4 with and the machining dataand inspection data and vice versa.

J. Third Embodiment

Next is described the third embodiment wherein the visual sensor 200notifies the PLC 100 of time information representing the timing atwhich the image data 50 created by taking an image of a workpiece 4 isstored in the data storage area 360, and the PLC 100 creates a mappingin the database 350 by including the time information in the informationtransmitted therefrom. In the third embodiment, the PLC 100 stillmanages the machining data, and the visual sensor 200 still manages theinspection data.

The image data 50 generated by the visual sensor 200 is transmitted tothe data storage area 360 for storage. The PLC 100 also generates aquery containing the machining data held in the PLC 100 and theinspection data output by the visual sensor 200, and transmits the queryto the database 350. The query the PLC 100 sends the database 350contains time information for specifying the image data 50 the visualsensor 200 requested to have stored.

That is, the visual sensor 200 (image processing device) stores eachpiece of image data 50 acquired in the data storage area 360, and sendsthe PLC 100 (control device) the time information representing thetiming at which the aforementioned image data 50 is stored.

On the other hand, the PLC 100 (control device) sends the database 350at least one of the information managed by the PLC 100 (typically,machining data) and the information managed by the visual sensor 200(typically, inspection data), together with the corresponding timeinformation. The database 350 creates the mappings for the correspondingimage data 50 on the basis of the time information transmitted from thePLC 100. That is, the database 350 (DB engine 352) references the datastorage area 360 on the basis of the time information provided in aquery received from the PLC 100, identifies the image data 50 stored atthe time indicated in the aforementioned time information, and uniquelymaps the query with the identified image data 50. In other words, thedatabase 350 creates the mapping between the machining data andinspection data, and the corresponding image data 50 and vice versa. Aconcrete example of the procedure is described with reference to asequence diagram.

FIG. 9 is a sequence diagram illustrating a processing procedure in theproduction system SYS according to a third embodiment. FIG. 9illustrates a specific data exchange procedure between the PLC 100, thevisual sensor 200, and the information gathering device 300 (thedatabase 350 and the data storage area 360).

Referring to FIG. 9, first, tracking is established between the PLC 100and the visual sensor 200 (sequence SQ300). Here, tracking isestablished in the same manner as in the above-described firstembodiment (sequence SQ100 in FIG. 7). Actual operations begin oncetracking is established.

The PLC 100 first detects the arrival of the workpiece 4 (sequenceSQ302), and initiates the execution of Fab Process 1 on the workpiece 4(sequence SQ304). The related machining data is generated and updatedwith the execution of Fab Process 1. Next, the PLC 100 initiates theexecution of Fab Process 2 on the workpiece 4 processed in Fab Process 1(sequence SQ306). The related machining data is generated and updatedwith the execution of Fab Process 2.

The visual sensor 200 detects the arrival of the workpiece 4 (sequenceSQ308); the visual sensor 200 initiates image processing on theworkpiece 4 on detecting the arrival thereof (sequence SQ310). Thisimage processing includes an inspection process, such as whether theworkpiece 4 is good or defective. The inspection process is where thevisual sensor 200 acquires the image data 50 by taking an image of theworkpiece 4, and the inspection data.

The visual sensor 200 transmits the acquired image data 50 to the datastorage area 360 (sequence SQ312). The data storage area 360 stores theimage data 50 transmitted from the visual sensor 200 (sequence SQ314).At this point, the filename for the image data 50 to be stored in thedata storage area 360 can be any desired name so long as the name doesnot already exist for another piece of image data 50.

At the same time, the visual sensor 200 sends the PLC 100 the timinginformation indicating the time at which the image data 50 was stored inthe data storage area 360 (sequence SQ316), and also sends the PLC 100the inspection data acquired from the inspection process (sequenceSQ318).

On receiving the inspection data from the visual sensor 200, the PLC 100uses the machining data acquired during Fab Process 1 and Fab Process 2,the inspection data from the visual sensor 200, and the time informationsent in advance to the visual sensor 200 to generate a query (sequenceSQ320), and sends the query to the database 350 (sequence SQ322). Notethat all the information is not necessarily transmitted using anindividual query; the information may be divided among and transmittedvia a plurality of queries.

The database 350 (DB engine 352) references the data storage area 360 onthe basis of the time information provided in a query sent from the PLC100 to identify the image data 50 stored at the time indicated by theaforementioned time information, and maps the query with the identifiedimage data 50 (sequence SQ324). In addition, the database 360 (DB engine352) updates the database so that the information related to the mappingis contained therein (sequence SQ324).

Hereafter, the processing in the sequences SQ302 to SQ324 is repeatedfor each of the workpieces 4. Executing these processes builds the sortof database illustrated in FIG. 3, and uniquely maps the correspondingimage data 50 for each of the workpieces 4 with the machining data andthe inspection data and vice versa.

K. Other Embodiments

The above-described first through third embodiments exemplifyconfigurations that track the workpieces 4 on the basis of somerelationship between the PLC 100 and the visual sensor 200 (e.g., anoffset value). However, workpiece tracking is not limited to thismethod, and any desired configuration may be adopted. That is, as longas there is a mapping between the machining data managed by the PLC 100,and the inspection data managed by the visual sensor 200 and vice versafor the same workpiece 4, the methods for implementing this kind ofmapping may include assigning each of the workpieces 4 some kind ofidentification such as a bar code, a wireless tag, or the like, andcreating mappings between the data on the basis of the identificationinformation.

L. Modification Examples

When the above-described embodiments are implemented using a generalpurpose computer, an operating system (OS) that provides the basicfunctions of a computer may be installed on the general-purpose computerin addition to programs that provide the various features of theembodiments. In this case, a program according to the embodiments maycall the required program modules provided as a part of the OS in aprescribed sequence and/or timing. In other words, there are cases wherea program according to the embodiments can run in collaboration with theOS without containing the above-mentioned program modules. Thus, aprogram according to the embodiments may exclude such kinds of programmodules that are a part of the OS.

Additionally a program according to the embodiments may be built-in as apart of another program. Even in this case, the program itself may runin collaboration with said other program without containing the modulesthat are available in the other program with which the program iscombined. That is, a program according to the embodiments may bebuilt-in as a part of this kind of other program.

Finally, the functions provided by executing the program may beimplemented in whole or in part as a dedicated hardware circuit.

M. Advantages

According to the embodiments, a unique mapping can be automaticallycreated for the image data 50 corresponding to a workpiece, themachining data, and the inspection data and stored in a database.Therefore, even with a multitude of analysis parameters available, i.e.,even with a large amount of data for analysis, analysis can be moreefficiently performed on the machining data and the inspection dataalong with the corresponding image data.

All aspects of the embodiments disclosed should be considered merelyexamples and not limitations as such. The scope of the present inventionis not limited to the above description but to the scope of the claims,and is intended to include all equivalents and modifications allowableby the scope of the claims.

The invention claimed is:
 1. A control system configured to gather andanalyze, for an object for inspection, information to be stored in adatabase and image data to be stored in a data storage area duringindependent operations and collaborative operations, the control systemcomprising: a control device comprising a first processor that executesone or more programs causing the control device to perform operationscomprising controlling a process comprising a plurality of stagesexecuted on the object for inspection; and an image processing devicearranged in communication with the control device, the image processingdevice comprising a second processor that executes one or more programsfor causing the image processing device to perform, for the plurality ofstages, the independent operations, the independent operationscomprising: acquiring image data by taking an image of the object forinspection, processing the image data acquired from taking the image ofthe object for inspection, the processing of the image data comprisingdetermining whether the image data of the object for inspectionindicates that the object for inspection is defective, and causing theimage data acquired by the image processing device to be stored in thedata storage area according to designation information specifying astorage destination in the data storage area designated for the objectfor inspection; wherein the information to be stored in the databasecomprises: an attribute value associated with the object for inspection,results information comprising a result from the processing of the imagedata of the object for inspection, and the designation information; thefirst processor of the control device and the second processor of theimage processing device perform the collaborative operations for storingthe information in the database; the attribute value is managed by thefirst processor of the control device; the results information ismanaged by the second processor of the image processing device; thedesignation information is managed by one of: the first processor of thecontrol device; and the second processor of the image processing deviceto enable the collaborative operations such that: in response to thefirst processor of the control device managing the designationinformation, the collaborative operations comprise the second processorof the image processing device sending the results information to thefirst processor of the control device and receiving the designationinformation from the first processor of the control device, and inresponse to the second processor of the image processing device managingthe designation information, the collaborative operations comprise thesecond processor of the image processing device sending the resultsinformation and the designation information to the first processor ofthe control device; the collaborative operations further comprise thefirst processor of the control device creating a unique mapping of theinformation associated with the object for inspection to be stored inthe database and the image data corresponding to the object forinspection stored in the data storage area for the plurality of stagesof the process based on the designation information; and the uniquemapping facilitates retrieval of the image data corresponding to theobject for inspection from the data storage area and retrieval of theinformation associated with the object for inspection from the databaseat each of the plurality of stages of the process.
 2. The control systemaccording to claim 1, wherein the first processor of the control deviceperforms operations further comprising providing the second processor ofthe image processing device with at least one of: identificationinformation; and location information for the image data acquired by theimage processing device as the designation information, theidentification information created in response to the storing of theimage data in the data storage area, and the location informationcomprising a location in the data storage area where the image data isstored; and.
 3. The control system according to claim 1, wherein thesecond processor of the image processing device performs operationsfurther comprising causing the image data acquired by the imageprocessing device to be stored in the data storage area in accordancewith a preliminarily determined rule, and providing the designationinformation corresponding to the stored image data to the firstprocessor of the control device.
 4. The control system according toclaim 2, wherein the first processor of the control device performsoperations further comprising: sending at least one of: informationmanaged by the control device; and information managed by the imageprocessing device to the database, together with the designationinformation for specifying the storage destination of the image datastored in the data storage area.
 5. The control system according toclaim 1, wherein the second processor of the image processing deviceperforms operations further comprising providing time information to thefirst processor of the control device comprising a time at which theimage data was stored in the data storage area.
 6. The control systemaccording to claim 5, wherein the first processor of the control deviceperforms operations further comprising sending at least one of:information managed by the control device; and information managed bythe image processing device to the database together with the timeinformation; and causing a process to be executed in the database tocreate a mapping of the information sent by the first processor of thecontrol device and the image data corresponding to the object forinspection based on the time information.
 7. A control device configuredto communicate with a database for storing information associated withan object for inspection, a data storage area for storing image data,and an image processing device, the control device comprising a firstprocessor that executes one or more programs causing the control deviceto perform operations comprising controlling a process comprising aplurality of stages executed on the object for inspection; wherein thefirst processor is configured to perform operations comprising:communicating with the image processing device, the image processingdevice comprising a second processor that executes one or more programsfor causing the image processing device to perform, for the plurality ofstages, independent operations comprising: acquiring image data bytaking an image of the object for inspection, processing the image dataacquired from taking the image of the object for inspection, theprocessing of the image data comprising determining whether the imagedata of the object for inspection indicates that the object forinspection is defective, and causing the image data acquired by theimage processing device to be stored in the data storage area accordingto designation information specifying a storage designation in the datastorage area designated for the object for inspection; the informationto be stored in the database comprises: an attribute value associatedwith the object for inspection, results information comprising a resultfrom the processing of the image data of the object for inspection, andthe designation information; the first processor of the control deviceand the second processor of the image processing device performcollaborative operations for storing the information in the database;the attribute value is managed by the first processor of the controldevice; the results information is managed by the second processor ofthe image processing device; the designation information is managed byone of: the first processor of the control device; and the secondprocessor of the image processing device to enable the collaborativeoperations such that: in response to the first processor of the controldevice managing the designation information, the collaborativeoperations comprise the second processor of the image processing devicesending the results information to the first processor of the controldevice and receiving the designation information from the firstprocessor of the control device, and in response to the second processorof the image processing device managing the designation information, thecollaborative operations comprise the second processor of the imageprocessing device sending the results information and the designationinformation to the first processor of the control device; thecollaborative operations further comprise the first processor of thecontrol device creating a unique mapping of the information associatedwith the object for inspection to be stored in the database and theimage data corresponding to the object for inspection stored in the datastorage area for the plurality of stages of the process based on thedesignation information; and the unique mapping facilitates retrieval ofthe image data corresponding to the object for inspection from the datastorage area and retrieval of the information associated with the objectfor inspection from the database at each of the plurality of stages ofthe process.
 8. An image processing device configured to communicatewith a database for storing information associated with an object forinspection, a data storage area for storing image data, and a controldevice, the image processing device comprising a second processor thatexecutes one or more programs for causing the image processing device toperform operations comprising: communicating with the control device,the control device comprising a first processor that executes one ormore programs causing the control device to perform operationscomprising controlling a process comprising a plurality of stagesexecuted on the object for inspection, and for the plurality of stages,independent operations comprising: acquiring image data by taking animage of the object for inspection; processing the image data acquiredfrom taking the image of the object for inspection, the processing ofthe image data comprising determining whether the image data of theobject for inspection indicates that the object for inspection isdefective; and causing the image data acquired by the image processingdevice to be stored in the data storage area according to designationinformation specifying a storage destination in the data storage areadesignated for the object for inspection; wherein the information to bestored in the database comprises: an attribute value associated with theobject for inspection, results information comprising a result from theprocessing of the image data of the object for inspection, and thedesignation information; the first processor of the control device andthe second processor of the image processing device performcollaborative operations for storing the information in the database;the attribute value is managed by the first processor of the controldevice; the results information is managed by the second processor ofthe image processing device; the designation information is managed byone of: the first processor of the control device; and the secondprocessor of the image processing device to enable the collaborativeoperations such that: in response to the first processor of the controldevice managing the designation information, the collaborativeoperations comprise the second processor of the image processing devicesending the results information to the first processor of the controldevice and receiving the designation information from the firstprocessor of the control device, and in response to the second processorof the image processing device managing the designation information, thecollaborative operations comprise the second processor of the imageprocessing device sending the results information and the designationinformation to the first processor of the control device; thecollaborative operations further comprise the first processor of thecontrol device creating a unique mapping of the information associatedwith the object for inspection to be stored in the database and theimage data corresponding to the object for inspection stored in the datastorage area for the plurality of stages of the process based on thedesignation information; and the unique mapping facilitates retrieval ofthe image data corresponding to the object for inspection from the datastorage area and retrieval of the information associated with the objectfor inspection from the database at each of the plurality of stages ofthe process.
 9. The image processing device according to claim 8,wherein the second processor of the image processing device isconfigured to perform operations further comprising sending timeinformation representing a time at which the image data was stored inthe data storage area to the first processor of the control device. 10.A control method for a control system configured to gather and analyze,for an object for inspection, information to be stored in a database andimage data to be stored in a data storage area, the control methodcomprising: controlling a process comprising a plurality of stagesexecuted on the object for inspection by a control device; taking animage of the object by an image processing device arranged incommunication with the control device; processing, by the imageprocessing device, the image data, the processing of the image datacomprising determining whether the image data of the object forinspection indicates that the object is defective; storing, by the imageprocessing device, the image data in the data storage area according todesignation information specifying a storage destination in the datastorage area designated for the object for inspection; and performing,by the control device and the image processing device, collaborativeoperations for storing the information in the database, wherein theinformation to be stored in the database comprises: an attribute valueassociated with the object for inspection, results informationcomprising a result from the processing of the image data of the objectfor inspection, and the designation information; the attribute value ismanaged by the control device; the results information is managed by theimage processing device; the designation information is managed by oneof: the control device; and the image processing device to enable thecollaborative operations such that: in response to the control devicemanaging the designation information, the collaborative operationscomprise the image processing device sending the results information tothe control device and receiving the designation information from thecontrol device, and in response to the image processing device managingthe designation information, the collaborative operations comprise theimage processing device sending the results information and thedesignation information to the control device; the collaborativeoperations further comprise the control device creating a unique mappingof the information associated with the object for inspection to bestored in the database and the image data corresponding to the objectfor inspection to be stored in the data storage area for the pluralityof stages of the process based on the designation information; and theunique mapping facilitates retrieval of the image data corresponding tothe object for inspection from the data storage area and retrieval ofthe information associated with the object for inspection from thedatabase at each of the plurality of stages of the process.